Skip to content

[CICD] Add KunlunXin unit and MCore integration test support - #94

Merged
Darryl233 merged 7 commits into
flagos-ai:mainfrom
BrianPei:Kunlun-dev
Aug 11, 2026
Merged

[CICD] Add KunlunXin unit and MCore integration test support#94
Darryl233 merged 7 commits into
flagos-ai:mainfrom
BrianPei:Kunlun-dev

Conversation

@AlexMa616

Copy link
Copy Markdown
Collaborator

Description

Summary

This PR adds dedicated KunlunXin CI support for TransformerEngine-FL using the existing vendor.kunlunxin backend.

Changes

  • Added the KunlunXin platform configuration, environment setup script, and test workflow.
  • Added KunlunXin launchers for:
    • PyTorch debug tests
    • PyTorch unit tests
    • Distributed tests
    • ONNX export tests
    • MCore integration tests
  • Added an explicit setup check to ensure:
    • transformer_engine_klx_torch can be imported.
    • The KunlunXin backend is available.
    • generic_gemm selects vendor.kunlunxin instead of falling back to another backend.
  • Kept KunlunXin-specific environment handling and compatibility logic outside the common workflows.
  • Added generic MCore entrypoint and CUDA-default controls for platform-specific launchers.
  • Pinned the Megatron-LM-FL revision used by the MCore integration test.
  • Enabled coverage collection and aggregation.

No TransformerEngine core source, upstream test cases, other platform configurations, or third-party submodules are changed.

Testing

The previous self-hosted KunlunXin workflow run passed:

  • PyTorch debug tests
  • PyTorch unit tests
  • Distributed tests
  • ONNX export tests
  • Coverage aggregation
  • MCore integration tests

The final cleaned branch additionally passed shell syntax, YAML parsing, embedded Python syntax, configuration path, and diff consistency checks.

@Darryl233 Darryl233 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@Darryl233
Darryl233 merged commit 94c7007 into flagos-ai:main Aug 11, 2026
75 of 80 checks passed
Caozhou1995 pushed a commit to Caozhou1995/TransformerEngine-FL that referenced this pull request Aug 11, 2026
Integrates upstream NVIDIA/TransformerEngine v2.17 (commit 2e559f0) into
flagos-ai/TransformerEngine-FL:main, preserving all multi-backend plugin work.

- Upstream v2.14 -> v2.17 integration (EP, MXFP8/NVFP4, fused router/MLP)
- 25 conflict files resolved, fork patches preserved (torch_nv, te_device_type,
  OpManager dispatcher, 9-backend CI/CD)
- P0: removed 2 deleted C++ bindings (convert_host_pointers_to_tensor,
  get_device_pointer_for_data_and_scales) from plugin + 6 vendors
- Enum sync: DType assert ==->subset, NVTE_QKV_Format/Layout members
- Device abstraction re-injected (quantized_tensor.py make_empty etc.)
- Includes flagos-ai#94 KunlunXin CICD support
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants